Referring to FIG. 1, an interline charge coupled device (CCD) image sensor 10 is comprised of an array of photodiodes 20. The photodiodes are covered by color filters to allow only a narrow band of light wavelengths to generate charge in the photodiodes. Referring to FIG. 2, typically image sensors have a pattern of three or more different color filters arranged over the photodiodes in a 2×2 sub array as shown in FIG. 2. For the purpose of a generalized discussion, the 2×2 array is assumed to have four colors, A, B, C, and D. The most common color filter pattern used in digital cameras is the Bayer pattern, in which color A is red, color B and C are green, and color D is blue.
Referring back to FIG. 1, image readout of the photo-generated charge begins with the transfer of some or all of the photodiode charge to the vertical CCD (VCCD) 30. In the case of a progressive scan CCD, every photodiode simultaneously transfers charge to the VCCD 30. In the case of a two field interlaced CCD, first the even numbered photodiode rows transfer charge to the VCCD 30 for first field image readout, then the odd numbered photodiode rows transfer charge to the VCCD 30 for second field image readout. Interlaced CCDs are not limited to two-field read out. Four or more interlaced fields are also commonly used.
Charge in the VCCD 30 is read out by transferring all columns in parallel one row at a time into the horizontal CCD (HCCD) 40. The HCCD 40 then serially transfers charge to an output amplifier 50.
FIG. 1 shows an array of only 24 pixels. Many digital cameras for still photography employ image sensors having millions of pixels. A 10-megapixel image sensor would require at least ⅓ second to read out at a 40 MHz data rate. This is not suitable if the same camera is to be used for recording video. A video recorder requires an image read out in 1/30 second. The shortcoming to be addressed by the present invention is how to use an image sensor with more than 1 million pixels as both a high quality digital still camera and 30 frames/second video camera.
The prior art addresses this problem by providing a video image at a reduced resolution (typically 640×480 pixels). For example, an image sensor with 3200×2400 pixels would be have only every fifth pixel read out as described in U.S. Pat. No. 6,342,921. This is often referred to as sub-sampling, or sometimes as thinned out mode or skipping mode. The disadvantage of sub-sampling the image by a factor of 5 is only 4% of the photodiodes are used. A sub-sampled image suffers from reduced photosensitivity and alias artifacts. If a sharp line focused on the image sensor is only on the un-sampled pixels, the line will not be reproduced in the video image. Other sub-sampling schemes are described in U.S. Pat. Nos. 5,668,597 and 5,828,406.
The prior art, including U.S. Pat. No. 6,661,451 or US Patent Application Publication 2002/0135689A1, attempts to resolve the problems of sub-sampling by summing pixels together. However, this prior art still leaves some pixels un-sampled.
US Patent Application Publication 2001/0010554A1 increases the frame rate by summing pixels together without sub-sampling. However, it requires a two field interlaced read out. It is more desirable to obtain a video image with progressive scan read out. Interlaced video acquires the two fields at different times. A moving object in the image will appear in different locations when each interlaced field is acquired.
Another disadvantage of the prior art is it only reduces the image resolution in the vertical direction. In the horizontal direction, the HCCD must still read out every pixel. Only reducing the image resolution through sub-sampling or other methods in the vertical direction does not increase the frame rate to 30 frames/second for very large (greater than 8 million pixels) image sensors.
US Patent Application Publication 2003/0067550A1 reduces the image resolution vertically and horizontally for even faster image readout. However, this prior art requires a striped color filter pattern (a 3×1 color filter array), which is generally acknowledged to be inferior to the Bayer or 2×2 color filter array patterns.
US Patent Application Publication 2004/0150733A1 addresses the disadvantages of sub-sampling by summing together groups of pixels in sub-arrays 2n+1 pixels square, where n is an integer. This only provides a means of summing pixel sub-arrays of an odd number of pixels. It also does not disclose the use of multiple horizontal CCDs for faster image read out at full resolution. The present invention discloses a means of summing together even numbered groups of pixels with multiple horizontal CCDs.
US Patent Application Publication 2005/0259171A1 provides a means of summing pixel sub-arrays with multiple horizontal CCDs. However, the horizontal CCD architecture does not provide a means of shifting charge packets in one horizontal CCD independently from a second horizontal CCD to provide proper summing of pixels such that the summed pixel pattern closely matches the Bayer color filter pattern. This deficiency arises from using only two horizontal CCD control gates that are common to all horizontal CCD registers. US Patent Application Publication 2005/0259171A1 also does not provide a means of reducing the total number of horizontal CCD clock cycles by a factor of two when summing pixel sub-arrays. That decreases the frame rate by a factor of two compared to the present invention.
In view of the deficiencies of the prior art, an invention is desired which is able to produce 30 frames/second video from a megapixel image sensor with a 2×2 color filter pattern while sampling 100% of the pixel array and reading out the video image progressive scan (non-interlaced) and also producing a new summed pixel array that closely matches the Bayer color filter pattern to enable use of standard Bayer color filter pattern interpolation and video compression hardware.